The combined effect of chemical and electrical synapses in small Hindmarsh-Rose 
neural networks on synchronisation and on the rate of information 
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In this work we studied the combined action of chemical and electrical synapses in small networks 
of Hindmarsh-Rose (HR) neurons on the synchronous behaviour and on the rate of information 
produced (per time unit) by the networks. We show that if the chemical synapse is excitatory, 
the larger the chemical synapse strength used the smaller the electrical synapse strength needed to 
achieve complete synchronisation, and for moderate synaptic strengths one should expect to find 
desynchronous behaviour. Otherwise, if the chemical synapse is inhibitory, the larger the chemical 
synapse strength used the larger the electrical synapse strength needed to achieve complete synchro- 
nisation, and for moderate synaptic strengths one should expect to find synchronous behaviours. 
Finally, we show how to calculate semi-analytically an upper bound for the rate of information 
produced per time unit (Kolmogorov-Sinai entropy) in larger networks. As an application, we show 
that this upper bound is linearly proportional to the number of neurons in a network whose neurons 
are highly connected. 

PACS numbers: 05.45.-a; 05.45. Gg; 05.45.Pq; 05.45.Xt 



I. INTRODUCTION 

Intercellular communication is one of the most impor- 
tant characteristics of all animal species because it makes 
the many components of such complex systems operate 
together. Among the many types of intercellular commu- 
nication, we are interested in the communication among 
brain cells, the neurons, that exchange information me- 
diated by chemical and electrical synapses [1]. 

The uncovering of the essence of behaviour and per- 
ception in animals and human beings is one of the main 
challenges in brain research. While the behaviour is be- 
lieved to be linked to the way neurons are connected 
(the topology of the neural network and the physical con- 
nections among the neurons), the perception is believed 
to be linked to synchronisation. This comes from the 
binding hypothesis [2] , which states that synchronisation 
functionally binds neural networks coding the same fea- 
ture or objects. This hypothesis raised one of the most 
important contemporary debates in neurobiology [3] be- 
cause desynchronisation seems to play an important role 
in perception as well. The binding hypothesis is mainly 
supported by the belief that a convenient environment 
for neurons to exchange information appears when they 
become more synchronous. 

Despite the explosive growth in the field of complex 
networks, it is still unclear for which conditions synchro- 
nisation implies information transmission and it is still 
unclear which topology favours the flowing of informa- 
tion. Additionally, most of the models being currently 
studied in complex networks consider networks whose 
nodes (such as neurons) are either linearly or non-linearly 
connected. But, recent works have shown that neu- 
rons that were believed to make only non-linear (chem- 



ical) synapses make also simultaneously linear (electri- 
cal) synapses [4-8]. To make the scenario even more 
complicated, neurons connect chemically in an excita- 
tory and/or an inhibitory way. In this work, we aim to 
study the relationship between synchronisation and in- 
formation transmission in such neural networks, whose 
neurons are simultaneously connected by chemical and 
electrical synapses. 

The electrical synapse is the result of the potential dif- 
ference between the neurons and causes an immediate 
physiological response of the latter one, linearly propor- 
tional to the potential difference. The chemical synapse 
is mediated by the exchange of neurotransmitters from 
the pre to the postsynaptic neuron and can only be re- 
leased once the presynaptic neuron membrane achieves 
a certain action potential. The chemical interaction is 
described by a nonlinear function [9] . 

While the electrical synapses between neurons is lo- 
calised in the neuron cell and therefore it is a local con- 
nection, the chemical synapse is in the neuron axon and 
is therefore mainly responsible for the non-local nature 
of the synapses. 

Chemical synapses can be inhibitory and excitatory. 
When an inhibitory neuron spikes (the pre-synaptic neu- 
ron) , a neuron connected to it (the post-synaptic neuron) 
is prevented from spiking. As shown in Ref. [10], inhibi- 
tion promotes synchronisation. When an excitatory neu- 
ron spikes, it induces the post-synaptic neuron to spike. 
Several types of synchronisation were found in networks 
of chaotic neurons coupled with only electrical synapses. 
One can have complete synchronisation, generalised syn- 
chronisation and phase synchronisation, the latter ap- 
pearing for small synapse strength [11]. Complete syn- 
chrony strongly depends on the network structure and 
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the number of cells. In networks of chemically coupled 
neurons [12], the net input a neuron receives from synap- 
tic neurons emitting synchronised spikes is proportional 
to the number of connected units. Hence, for chemical 
synapses, if all the nodes in the network have the same de- 
gree, synchronisation will be enhanced; if different nodes 
have different degrees, synchronisation will be hampered 
[13]. In fact, Ref. [14] has shown analytically that the 
stability of the completely synchronous state in such net- 
works only depends on the number of signals each neuron 
receives, independent of all other details of the network 
topology. 

The most obvious possible role of electrical synapses 
within networks of inhibitory neurons is to couple the 
membrane potential of connected cells, leading to an in- 
crease in the probability of synchronised action poten- 
tials. This synchronous firing could coordinate the ac- 
tivity of other cortical cell populations. For example, 
it has been reported that the introduction of electrical 
synapses among GABAergic neurons that are also chem- 
ically connected can promote oscillatory rhythmic activ- 
ity [6]. These possibilities have been addressed experi- 
mentally by several investigators and have been reviewed 
recently [7, 8, 15]. 

Motivated by these observations and also by the fact 
that the behaviour of micro-circuitry in the cerebral cor- 
tex is not well understood, we analyse the combined ef- 
fect of these two types of synapses on the stability of 
the synchronous behaviour and on the information trans- 
mission in small neural networks. In order to deal with 
this problem analytically we consider idealistic networks, 
composed of equal neurons with mutual connections of 
equal strengths (see Sec. II). A basic assumption char- 
acterising most of the early works on synchronisation 
in neural networks is that, by adding a relatively small 
amount of electrical synapse to the inhibitory synapse, 
one can increase the degree of synchronisation far more 
than a much larger increase in inhibitory conductance 
[16, 17]. 

Our results agree with this finding in the sense that for 
larger inhibitory synaptic strengths complete synchro- 
nisation can only be achieved if the electrical synapse 
strength is larger than a certain amount. But in contrast, 
we found that for moderate inhibitory synaptic strengths, 
the larger the chemical synapse strength is the larger the 
electrical synapse strength needs to be to achieve com- 
plete synchronisation. Additionally, we introduce in this 
work analytical approaches to understand when complete 
synchronisation should be expected to be found and what 
is the relation of that with the amount of information 
produced by the network. 

Information is an important concept [18]. It measures 
how much uncertainty one has about an event before it 
happens. It is a measure of how complex a system is. 
Very complicated and higher dimensional systems might 
be actually very predictable, and as a consequence the 
content of information of such a system might be very 
limited. But measuring the amount of information is 



something difficult to accomplish. Normally, there is 
always some bias or error on the calculation of it [19], 
and one has to rely on alternative approaches. Measur- 
ing the Shannon entropy of a chaotic trajectory is ex- 
tremely difficult because one has to calculate an integral 
of the probability density of a fractal chaotic set. But 
for chaotic systems that have absolutely continuous con- 
ditional measures, one can calculate Shannon's entropy 
per unit of time, a quantity known as Kolmogorov-Sinai 
(KS) entropy [20], by summing all the positive Lyapunov 
exponents [21]. A system that has absolutely continuous 
conditional measures is a system whose trajectory con- 
tinuously distribute along unstable directions. More pre- 
cisely, systems whose trajectories continuously distribute 
along unstable manifolds at points that have positive 
probability measure. These systems form a large class of 
nonuniformly hyperbolic systems [22]: the Henon fam- 
ily; Henon-like attractor arising from homoclinic bifurca- 
tions; strange attractors arising from Hopf Bifurcations 
(e.g. Rossler oscillator); some classes of mechanical mod- 
els with periodic forcing. The result in Ref. [21] ex- 
tends a previous result by Pesin [23] that demonstrated 
that for hyperbolic maps, the KS entropy is equal to the 
sum of the positive Lyapunov exponents. We are not 
aware of any rigorous result proving the equivalence of 
the KS entropy and the sum of Lyapunov exponent for 
the Hindmarsh-Rose neural model neither to a network 
constructed with them. But the chaotic attractors aris- 
ing in this neuron model are similar to the ones appearing 
from Homoclinic bifurcations. Additionally, for two cou- 
pled neurons, we show in Sec. VII (using the non-rigorous 
methods described in Appendix XI) that a lower bound 
estimation of the KS entropy is indeed close to the sum of 
all the positive Lyapunov exponents. Despite the lack of 
a rigorous proof, we will assume that the results in Refs. 
[21, 22] apply in here in the sense that the sum of the 
positive Lyapunov exponents provide a good estimation 
for the KS entropy. 

The KS entropy for chaotic networks has another im- 
portant meaning. It provides one the so called network 
capacity [11], the maximal amount of information that 
all the neurons in the network can simultaneously pro- 
cess (per unit of time). A network that produces infor- 
mation at a higher rate is more unpredictable and more 
complex. Arguably, the network capacity is an upper 
bound for the amount of information that the network is 
capable of processing from external stimuli. In Ref. [11] 
we discuss a situation were that is indeed the case. 

To understand the scope of this paper and the methods 
used, we first justify the chosen network topologies in Sec. 
II. Then, in Sec. Ill, we describe the dynamical system of 
our network and derive the variational equations of it in 
the eigenmode form, a necessary analytical tool in order 
to be able to study the onset of complete synchronisation 
(CS) and to calculate the rate of information produced 
by the network. Complete synchronisation happens when 
the trajectories of all neurons are equal. 

Our main results can be summarised as in the follow- 
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ing: 

• We show (Sees. IV and V) how one can cal- 
culate the synaptic strengths (chemical and elec- 
trical) necessary for a network of N neurons to 
achieve complete synchronisation when one knows 
the strengths for which two mutually coupled neu- 
rons become completely synchronous. 

• We show numerically (Sec. VI) parameter space 
diagrams indicating the electrical and chemical 
synapse strengths responsible to make complete 
synchronisation to appear in different networks. 
The analytical derivation from Sec. V are found 
to be sufficiently accurate. There are two sce- 
narios for the appearance of complete synchroni- 
sation for inhibitory networks. If the chemical 
synapse strength is small, the larger the chemi- 
cal synapse strength used the larger the electri- 
cal synapse strength needed to be to achieve com- 
plete synchronisation. Otherwise, if the chemical 
synapse strength is large, complete synchronisation 
appears if the electrical synapse strength is lager 
than a certain value. In excitatory networks both 
synapses work in a constructive way to promote 
complete synchronisation: the larger the chemi- 
cal synapse strength is the smaller the electrical 
synapse strength needs to be to achieve complete 
synchronisation. 

• We show (Sees. VII) that the sum of the positive 
Lyapunov exponents provides a good estimation for 
the KS entropy. Additionally, we show that there 
are optimal ranges of values for the chemical and 
electrical strengths for which the amount of infor- 
mation is large. 

• If complete synchronisation is absent, we show (Sec. 
VIII) that while in inhibitory networks one can typ- 
ically expect to find high levels of synchronous be- 
haviour, in excitatory networks one is likely to ex- 
pect desynchronous behaviour. 

• We calculate (Sec. IX) an upper bound for 
the rate of information produced per time unit 
(Kolmogorov-Sinai entropy) by larger networks us- 
ing the rate at which information is produced by 
two mutually coupled neurons. 

II. THE TOPOLOGY OF THE STUDIED 
NETWORKS 

In order to consider the combined action of these two 
different types of synapses, we need to consider in our 
theoretical approach idealistic networks, constructed by 
nodes possessing equal dynamics and particular coupling 
topologies such that a synchronisation manifold exists 
and CS is possible. If we had studied networks whose 
neurons were exclusively connected by electrical means, 



we could have considered networks with arbitrary topolo- 
gies. On the other hand, if we had studied networks 
whose neurons are exclusively connected by chemical 
means, we would have considered networks whose neu- 
rons receive the same number of chemical connections. 
These conditions are the same ones being usually made 
to study complete synchronisation in complex networks 
[14, 24]. 

In order to analytically study networks formed by neu- 
rons that make simultaneously chemical and electrical 
connections, we have not only to assume that the neurons 
have equal dynamics and that every neuron receives the 
same number of chemical connections coming from other 
neurons, but also that the Laplacian matrix for the elec- 
trical synapses (that provides topology of the electrical 
connections) and the Laplacian matrix for the chemical 
synapses commute, as we clarify later in this paper. Nat- 
urally, there is a large number of Laplacian matrices that 
commute. In this work we construct networks that are bi- 
ologically plausible. Since the electrical connection is lo- 
cal, we consider that neurons connect electrically only to 
their nearest neighbours. Since neurons connected chem- 
ically make a large number of connections (of the order of 
1000), it is reasonable to consider that for small networks 
the neurons that are chemically connected are fully con- 
nected, i.e., every neuron connects to all the other neu- 
rons. Notice however that while reciprocal connections 
are commonly found in electrically coupled neurons, that 
is not typical for chemically connected neurons. 

Since our small networks are composed of no more than 
8 neurons, we make an abstract assumption and admit 
another possible type of network in which neurons that 
are connected electrically can also make non-local con- 
nections, allowing them to become fully connected to the 
other neurons. Notice, however, that our theoretical ap- 
proach remains valid for larger networks that admit a 
synchronisation manifold. 

III. THE NETWORKS OF COUPLED NEURONS 
AND MASTER STABILITY ANALYSIS 

The dynamics of the Hindmarsh-Rose (HR) model for 
neurons is described by 

p = q-ap 3 + bp 2 -n + I ext 

q = c-dp 2 -q (1) 

n = r[s(p-p ) - n] 

where p is the membrane potential, q is associated with 
the fast current, Na + or K + , and n with the slow current, 
for example, Ca 2+ . The parameters are defined as a = 
1,6 = 3,c = l,d = 5,s = 4,r = 0.005, p = -1.60 and 
Iext =3.2 where the system exhibits a multi-time-scale 
chaotic behaviour characterised as spike-bursting. 

The dynamics of a neural networks of N neurons con- 
nected simultaneously by electrical (a linear coupling) 
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and chemical (a non-linear coupling) synapses is de- 
scribed by 

p\ = q, - ap\ + bp 2 -n, + I ext 

N N 

-9n(Pi ~ V syn ) C ij S (Pj) + m G ^ H fe) 

3=1 3 = 1 

q\ = c- dp 2 - q t (2) 
rii = r[s(pi - po) - Hi] 

— 1, ... ,7V, where N is the number of neurons. 

In this work we consider that H(pj) = pi. But we 
preserve the function H(pj) in our remaining analytical 
derivation to maintain generality. 

The chemical synapse function is modelled by the sig- 
moidal function 



S(Pj) = 



(3) 



with Q syn = —0.25, A = 10 and V syn = 2.0 for excitatory 
and V syn — —2.0 for inhibitory. For the chosen param- 
eters and all the networks that we have worked \pt\ < 2 
and the term (pi — V syn ) is always negative for excita- 
tory networks and positive for inhibitory networks. If 
two neurons are connected under an inhibitory (excita- 
tory) synapse then, when the presynaptic neuron spikes, 
it induces the postsynaptic neuron not to spike (to spike). 

The matrix describes the way neurons are electri- 
cally connected. It is a Laplacian matrix and therefore 
Gij = 0. The matrix describes the way neurons 
are chemically connected and it is an adjacent matrix, 
therefore = k, for all i. For both matrices, a pos- 

itive off-diagonal term placed in the line i and column j 
means that neuron i perturbs neuron j with an intensity 
given by <7/Gjj (or by g n Cij). Since the diagonal ele- 
ments of the adjacent matrix are zero, k represents the 
number of connections that neuron i receives from all the 
other neurons j in the network. This is a necessary con- 
dition for the existence of the synchronous solution [14] 
by the subspace P = P 1 = P 2 = .. = P N , Pi = (p it q it n l ). 

Under these assumptions and, as previously explained, 
we consider networks with three topologies: topology I, 
when all the neurons are mutually fully (all-to-all) con- 
nected with chemical synapses and mutually diffusively 
(nearest neighbours) connected with electrical synapses; 
topology II, when all the neurons are mutually fully 
connected with chemical synapses and mutually fully 
connected with electrical synapses; topology III, when 
all the neurons are mutually diffusively (nearest neigh- 
bours) connected with chemical and electrical synapses. 
We consider networks with 2, 4 and 8 neurons. By near- 
est neighbours, we consider that the neurons arc forming 
a closed ring. 

The synchronous solutions P = (p, q, n) take the form 



P = q-ap 3 + bp 2 -n + I ext - g n k(p - V syn )S(p) 
q = c — dp 2 — q (4) 
h = r[s(p - po) - n] 



The variational equation of the network in (2) [calculated 
around the synchronisation manifold (4)] is given by 



Spi = Sqi - 3ap 2 5pi + 2bpi8pi - Srii 



N 



-g n {pi - v syn )S (p) kSpt + Y GijSpj 



3 = 1 



N 



-kg n S(p)Spi + gi Y GijDK{p)5pi 

3 = 1 

8q t = 2dSpi - 5q t (5) 
Srii = r(sSpi — Srii) 

The matrix Cjj has been transformed to a Laplacian ma- 
trix by G = Cij — kl. DH(p) represents the derivative 
of H with respect to p, which in this work equals 1. 

The term S'(p) refers to the spatial derivative dS J^ 
and equals 



S'(P) 



[l + cxp-Mp-t-W)] 2 ' 



(6) 



Notice that if S(p) — 1 (what happens for p » <d syn ), 
then S'{p) = 0andifS*(p) = {p « e s2/ „),then S'(p) = 
0. S'(p) is not zero when the value of S(p) changes from 
1 to (and vice- versa) and p ~ & syn ■ 

Equation (5) is referred to as the variational equation 
and is often the starting point for determining whether 
the synchronisation manifold is stable. This equation 
is rather complicated since, given arbitrary synapses g n 
and gi, it can become quite higher dimensional. Also 
the coupling matrices G and G can be arbitrary making 
the situation to become even more complicated. How- 
ever, assuming that whenever there is a chemical synapse 
(and g n > 0), the matrices G and G commute, then 
the problem can be simplified by noticing that the ar- 
bitrary state 5X (where SX = (Spi, Sqi, Srii) is the de- 
viation of the ith vector state from the synchronisation 
manifold) can be written as SX = YliLi v i ® K i(t), with 
Ki(t) = (r]i,il>i,(pi). The Vj be the eigenvector and -fi and 
7, the corresponding eigenvalues for the matrices G and 
G respectively. So, if that is the case, by applying vj(t) 
(with vj(i) • Vi = Sij where Sij is the Kronecker delta), 
to the left (right) side of each term in Eq. (5) one finally 
obtains the following set of N variational equations in the 
eigenmode 



Vj = ( 2b P - 3ap 2 )r)j - ipj + ipj - T{p)r)j 
tpj = 2drij — il>j 

¥>3 = r ( sr l3 - <Pj) 

j = 1, 2, 3, ...N 
where the term T(p) is given by 



(7) 



T(p) = kg n S(p) - g n (V syn - p)S (p) (k + 7j) - gaj (8) 
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TABLE I: Values of 72 in absolute value and k for the con- 
sidered networks. 





all-to-all 


nearest-neighbour 


N = 2 
N = 4 
AT = 8 


72 = 2, fc=l 
72 = 4, fc=3 
72 = 8, k=7 


72 = 2, k=l 
72 = 2, fc=2 
72 = 0.585786402, k=2 



in which -fj (with 71 =0, and -fj <0, j > 2) are the eigen- 
values of G and jj are the eigenvalues of G. The eigen- 
values 7j are negative because the off-diagonal elements 
of G are positive. 

For networks with N — 2 we have that I72I = 2 and 
k = 1, meaning that the neurons are connected in an 
all-to-all fashion. For networks with N = 4, if the neu- 
rons are connected in an all-to-all fashion, we have that 
1 72 1 = 4 and k = 3 or if the neurons are connected 
with their nearest neighbours we have that |7 2 | = 2 and 
k = 2. For N = 8, (72) = 8 and k = 7 (all-to-all) 
and I72I = 0.585786402 and k = 2 (nearest- neighbour) . 
These values are placed in Table I for further reference. 

The previous equations are integrated using the 4th- 
order Rangc-Kutta method with a step size of 0.001. The 
calculations of the Lyapunov exponents are performed 
considering a time interval of 600 [sufficient for a neuron 
to produce approximately 600 spikes (p > 0)]. We dis- 
card a transient time of 300, corresponding to 300,000 
integrations. 



IV. STABILITY ANALYSIS 

The stability of the synchronisation manifold can be 
seen from the perspective of control [14, 25-27] by imag- 
ining that the term T(p) stabilises Eq. (7) at the origin. 
This term can be interpreted as the main gain of a feed- 
back control law u(t) = T(p)rjj such that r]j (resp. tpj 
and <fj ) tends to as t tends to infinity. In fact, the 
controlling force u{t) = T(p)rjj could be designed with 
no previous knowledge of the system under considera- 
tion assuming that it has a parametric dependence. A 
drawback of such a general control approach is that it 
leads to non-feedback control strategy, which have not 
guaranteed stability margins. More robust approaches 
for determining the structural stability of the synchroni- 
sation manifold of systems whose equations of motion are 
partially unknown have been recently developed [25-27]. 

In this work, however, we determine the stability of 
the synchronisation manifold from the master stability 
analysis of Refs. [14, 24]. A necessary condition for the 
linear stability of the synchronised state is that all Lya- 
punov exponents associated with jj and/or 7^ for each 
j = 2,3, ...,N (the directions transverse to the synchro- 
nisation manifold) are negative. This criterion is a neces- 
sary condition for complete synchronisation only locally, 
i.e. close to the synchronisation manifold. 



V. RESCALING OF EQS. (4) AND (7) 

When working with networks formed by nodes pos- 
sessing equal dynamical rules, we wish to predict the be- 
haviour of a large network from the behaviour of two 
coupled nodes. That can always be done whenever the 
equations of motion of the network can be rescaled into 
the form of the equations describing the two coupled 
nodes. That means that, given that two mutually cou- 
pled neurons completely synchronise for the electrical and 
chemical synapse strengths g*(N — 2) and g„(N = 2), 
respectively, then it is possible to calculate the synapse 
strengths g*(N) and 3^ (AT) for which a network com- 
posed by N nodes completely synchronises. 

In order to rescale the equations for the synchroni- 
sation manifold and for its stability, Eqs. (4) and (7), 
respectively, we need to preserve the form of these equa- 
tions as we consider different networks. 

Concerning Eq. (7), we need to show under which 
conditions it is possible to have T(p, N = 2) = T(p,N), 
where T is the term responsible to make the stability 
of the synchronisation manifold to depend among other 
things on the topology of the network and on the coupling 
function S(p). 

Notice that S(p) assumes for most of the time either 
the value or 1. For some short time interval S(p) 
changes its value from to 1 (and vice-versa) and at 
this time S'(p) is different from zero [see Eqs. (3) and 
(6)]. For that reason we will treat S'(p) as a small per- 
turbation in our further calculations and will ignore it, 
most of the times. That leave us with two relevant terms 
in both Eqs. (4) and (7) that need to be taken into con- 
sideration in our rescaling analyses. These terms are gi^fj 
and kg n S(p). While the first term comes from the elec- 
trical synapse, the second term comes from the chemical 
synapse. 

The first term depends on the eigenvalues of 
(which varies according to the number of nodes and the 
topology of the network) and on the synapse strength 
gi. If this term assumes a particular value for a given 
network, for another network one can suitably vary <?; in 
order for the whole term to assume this same value in the 
other network. So, the term gijj can always be rescaled 
by finding an appropriate value of g\. 

The rescaling of the second term, kg n S(p) is more com- 
plicated because it depends on the trajectory (p) of the 
attractor. Naturally, we wish to find a proper rescaling 
for the function S(p), which implies that the attractors 
appearing as solutions on the synchronisation manifold 
should present some kind of invariant property. 

In order to find such an invariant property, we study 
the time average (S(p)) of the function S(p) for attractors 
appearing as solutions of Eq. (4) for 5 network topolo- 
gies. In Fig. 1 we show in the boxes (A-E) the values 
of N,\^f2\, k and the type of topology considered in the 
networks of Figs. 2, 3, 4, and 5. 

The result for excitatory networks can be seen in 
Fig. 2(A-E), which shows this value as a function of 
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FIG. 1: The topology of the networks considered in Figs. 2, 
3, 4 and 5 and the values of N, I72I and k. 



kg n . Apart from some small differences, the function 
(S(p)) remains invariant for the different networks con- 
sidered. We identify two relevant values for (S(p)). Ei- 
ther (S(p)> w 0.9, for 5 „ < g n c) or (S(p)) = 0, for 

9n >gi C) - 9^ » 1-67. 

We also find an invariant curve of {S(p)) for inhibitory 
networks. In Fig. 3(A-E) we show this curve for the 
same networks of Fig. 2. For these networks, we define 
gffl ~ 1-5 as the value of g n for which the curve of (S(p)) 
reaches its maximum. In the considered inhibitory net- 
works, (S(p)) = 1 is a consequence of the fact that the 
neurons loose their chaotic behaviour and become a sta- 
ble limit cycle. Notice that the value of {S(p)) does not 
depend on the value of the electrical synapse strength 
gi. This is due to the fact that gi is not present in the 
equations for the synchronisation manifold [Eq. (4)]. 



FIG. 3: (A-E) The value of (S(p)) with respect to a rescaled 
chemical synapse strength kg n for inhibitory networks with a 
configuration shown in Figs. l(A-E). Initial conditions of the 
neurons are set to be equal (and gi=0). 



Let us rescale Eq. (4). First notice that the aver- 
age {(p — V syn )) has the same invariant properties of the 
average {S(p)). Then, we assume that both S(p) and 
(p — V S yn) make small oscillations around their average 
value. That implies that S(p)(p — V syn ) ~ (S(p)(p — 
V syn )). From Figs. 2 and 3 we have that the average 
{S(p, N)) can be written as a function of g n (N), as well 
as ((p — V syn ))- Therefore, we can write (S(p)(p— V syn )) 
as a function of g n (N). It is clear that the value of this 
average obtained for g n (N = 2) should be approximately 
equal to the value obtained for kg n {N), and so this av- 
erage function can be rescaled by kg n (N) ~ g n {N = 2). 
Therefore, Eq. (4) describing a large network can be 
rescaled into this same equation describing two mutually 
coupled neurons by 




FIG. 2: (A-E) The value of (S(p)) with respect to a rescaled 
chemical synapse strength kg n for excitatory networks with a 
configuration shown in Figs. l(A-E). Initial conditions of the 
neurons are set to be equal (and gi—O). 



9n(N) = 



9n(N = 2) 

k 



(9) 



Now, we need to show that it is also possible to do the 
same to Eq. (7), the equation responsible for the stability 
of the synchronous solution. 

Assuming again that S(p) make small oscillations 
around its average value allows us to write E(p, N) as 
a function of (S(p)} as in F(p, N) ^ kg n (N)(S(p, N)) - 
gi(N)"fj. Notice from Figs. 2 and 3 that the average 
(S(p,N)} can be written as a function of g n (N). In or- 
der to rescale Eq. (7), describing a network of N nodes 
in terms of a network of 2 nodes, we need to have that 
T(p, N) = T(p, N = 2) leading to 



kg n (N){S[g n (N)})- l2gi (N) 
g n (N = 2)(S[g n (N = 2)])+2 gi (N) 



(10) 



where we have considered only the second largest eigen- 
value 72, the one responsible for the stability of the syn- 
chronisation manifold; we have ignored terms that appear 
together with S' in F. 
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We make now a reasonable hypothesis that if a stable 
synchronous solutions for Eq. (4) exists for g n (N = 2) = 
g„(N = 2) (for a two mutually coupled neurons), then 
this same stable synchronous solution exists for kg* n (N) 
(for a network composed by N neurons mutually con- 
nected). This hypothesis is constructed from the obser- 
vation that equivalent attractors can be found in differ- 
ent networks if the rescaling in Eq. (9) is employed. We 
are assuming that if g„(N = 2) represents the chemical 
synapse strength for which complete synchronisation ap- 
pears in two mutually coupled neurons, then complete 
synchronisation would appear in a network of N nodes if 



9*n(N = 2) 



(11) 



If the previous hypothesis is satisfied, i.e. Eq. (11) is 
satisfied, we see from Figs. 2 and 3 that (S[g n (N)]) ~ 
(S[g n (N — 2)]) and assuming that these two averages are 
equal, then Eq. (10) takes us to 



9t(N) 



|72(A0I 



(12) 



where gf{N) represents the electrical synapse strength 
for which complete synchronisation occurs in a network 
composed by N neurons. 

In the following, we analyse two special cases of Eq. 
(10) when the function S(p) is constant and the previous 
approximations (expanding T around its average and that 
(S[g n (N)\) = {S[g n (N = 2)])) to arrive to Eqs. (11) and 
(12) are exact. 



A. Rescaling in excitatory networks (V 3yn = 2.0) 



in this section, our calculations provide a good estimation 
of what to expect from parameter spaces of larger net- 
works when the parameter space of two mutually coupled 
neurons is known. The parameter space is constructed 
by considering the synapses (gi, g n ) and they identify the 
regions where the state of complete synchronisation is 
stable. 

The stability is determined from Eqs. (7), by verifying 
whether there are no lyapunov exponents associated with 
transversal directions to the synchronisation manifold. 
These exponents are numerically obtained, without any 
approximation. 

In Fig. 4, we show in black the synchronous regions 
(all transversal conditional exponents are negative) for 
the excitatory networks and in Fig. 5 the same network 
topologies but for inhibitory networks. To simplify the 
understanding of these two figures, in Fig. 1 we show 
in boxes (A-E) the values of N, I72I, k and the type of 
topology considered in the networks of Figs. 4(A-E) and 
5 (A-E). The values of gi and g n were rescaled by using 
Eqs. (11) and (12). As expected, in excitatory networks 
our rescaling works very well and roughly in inhibitory 
networks. So, the vertical axis of Figs. 4(B-E) and 5(B- 
E) show the quantity kg n (N) and the horizontal axis of 
these same figures show the quantity . 

To assist the analysis of the parameter spaces, imagine 
a curve S that is the border between the regions defin- 
ing parameters for which the synchronisation manifold is 
unstable (white regions) and regions defining parameters 
for which the synchronisation manifold is stable (black re- 
gions). There are four main characteristics in these two 
types (excitatory and inhibitory) of networks concerning 
the occurrence of complete synchronisation. 



Case 1: A large chemical synapse strength, kg n (N) > 
g„ \ with gfff* ESl.67, makes for all the time p < O, lead- 
ing to S(p) = and S'(p)=0 (see Fig. 2). The neurons 
become completely synchronous to a stable equilibrium 
point. 

B. Rescaling in inhibitory networks (V sy n = —2.0) 

Case 2: a large chemical synapse strength, kg n (N) > 
g„ \ with g^ ~ 1.50, makes for all the time p > and 
as a consequence S(p) — 1 and S(p)' — (see Fig. 3). 
The neurons become completely synchronous to a limit 
cycle. 

VI. COMBINED EFFECT OF THE CHEMICAL 
AND ELECTRICAL SYNAPSES ON THE 
SYNCHRONOUS BEHAVIOUR 

The analytical derivations done in the previous section 
are approximations, except for some special values of the 
synaptic strengths (case 1 and 2). However, as we show 




FIG. 4: Excitatory networks. Black points represent values 
of the synapse strengths for which all transversal conditional 
exponents are negative. In (B-E) the horizontal axis represent 
<7;(iV)|72(iV)|/2 and the vertical axis kg n - Initial conditions 
of the neurons are set to be equal. 



• In excitatory networks, the electrical and the 
chemical synapses act in a combined way to foster 
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FIG. 5: Inhibitory networks. Black points represent values 
of the synapse strengths for which all transversal conditional 
exponents are negative. In (B-E) the horizontal axis represent 
</;(A r )|72(A r )|/2 and the vertical axis kg n . Initial conditions 
of the neurons are set to be equal. 



for similar synaptic strengths for which the synchronisa- 
tion manifold is stable. 

If the neurons are set with sufficiently different ini- 
tial conditions, and we construct parameter spaces that 
represent synaptic strengths for which CS takes place, 
we would have obtained parameter spaces with similar 
structure as the one observed in Figs. 4 and 5. How- 
ever, the network can become completely synchronous to 
other synchronous solutions of Eq. (4), different from 
the synchronous solutions observed for the parameters 
used to make Figs. 4 and 5. In other words, parameter 
spaces that show CS in networks whose neurons are set 
with different initial conditions constructed for the same 
synaptic strengths and networks considered in Figs. 4 
and 5 would present additional black points in the white 
areas of Figs. 4 and 5. 



VII. COMBINED EFFECT OF THE CHEMICAL 
AND ELECTRICAL SYNAPSES ON THE 
AMOUNT OF INFORMATION 



synchronisation. The neurons become completely 
synchronous to a stable equilibrium point. The 
asynchronous neurons (white regions) are chaotic. 
The curve X would look like a diagonal line with a 
negative slope. Such a curve could be defined by 
an equation similar to kg(N) + j29l ~ C, C being 
a function that is approximately constant (see Fig. 

4) . 

• In excitatory networks, with kg n (N) >1.67, 
Neurons are completely synchronous to a stable 
equilibrium point (see Fig. 4). 

• In inhibitory networks, with kg n (N) <5, the 
larger the chemical synapse strength is the larger 
the electrical synapse strength needs to be to 
achieve complete synchronisation. Neurons be- 
come completely synchronous to either a limit cycle 
(large chemical synapse strength) or to a chaotic 
attractor (small chemical synapse strength). The 
curve S would look like a diagonal line with a pos- 
itive slope. Such a curve could be defined by an 
equation similar to kg(N) — 72.9; ~ C, C being a 
function that is approximately constant (see Fig. 

5) . 

• In inhibitory networks, for large values 

of kg n (N), complete synchronisation appears for 
725; > C and neurons become completely syn- 
chronous to a stable limit cycle, which is unstable 
if 729/ < C. The curve X would look like a straight 
vertical line. Such a curve could be defined by an 
equation similar to 725/ ~ C. C being a function 
that is approximately constant (see Fig. 5). 

If the neurons are set with different initial conditions, 
but sufficiently close, complete synchronisation is found 
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FIG. 6: [Color Online] We show the value of the sum of all the 
positive Lyapunov exponents Hl in black line and an estima- 
tion of the lower bound for the KS entropy in filled squares 
(red line online) for two mutually chemically coupled neurons 
under an excitatory synapse (A) and an inhibitory synapse 
(B), as we vary the chemical synapse strength. We consider 
a constant electrical synapse of strength <?;=0.1. Initial con- 
ditions are not equal. 

First, we calculate the sum of all the positive Lyapunov 
exponents of the attractor obtained from integrating the 
neural network [Eq. (2)] and represent it by Hl- The 
Lyapunov exponents are calculated from the variational 
equation of the network in Eq. (2). As previously dis- 
cussed, it is reasonable to assume that Hl ~ Hks, where 
Hks represents the KS entropy [20], which measures the 
amount of information (Shannon's entropy) produced per 
time unit. 

In Figs. 6(A-B) we show in the thin line Hl for 
two mutually chemically and electrically coupled neurons 
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(<7i=0.1) for excitatory synapse (A) and for inhibitory 
synapse (B). To confirm that the sum of the positive Lya- 
punov exponents have an entropic meaning for the stud- 
ied Hindmarsh-Rose neuron model, we have estimated a 
lower bound for the KS entropy represented by the tick 
line with filled squares (red online) in Fig. 6(A-B). 

We see that for both cases, as one increases the synap- 
tic strength, Hl decreases. For the excitatory case, for 
g n > 1.52, the neurons trajectories go to an equilibrium 
point and we obtain H L = 0. If H L = 0, that means that 
there are no positive Lyapunov exponents and therefore 
no chaos. The maximal value of Hl, calculated vary- 
ing the synaptic strengths, is almost equal for both types 
of synapses. One sees that there is a range of strength 
values in both figures within which Hl is large. For ex- 
ample, in (A) Hl is large for g n G [0.7, 1.2] and in (B) 
Hl is large for g n G [0.3, 0.7]. This was also observed in 
3D parameter space diagrams (not shown in here) that 
show the value of Hl versus g n and gi. These diagrams 
indicate that there is an optimal range of values for g n 
and gi for which Hl remains large. 

The reason we have shown results for two coupled neu- 
rons is because for such a configuration a lower bound 
estimation of the KS entropy can be calculated by en- 
coding the trajectory into a binary symbolic sequence. 
Since the sequence is binary, this method is only capa- 
ble of measuring an information rate that is less or equal 
than lbit/symbol or lbit/unit of time. Since that for two 
coupled neurons, Hl < lbit/unit of time, and assuming 
that Hl is a good estimation for Hks, then the employed 
method to calculate a lower bound of the KS entropy is 
appropriate. The details of this estimation can be seen 
in Appendix XI. 

Notice that in Fig. 6(A-B) for g n «0 (as well as in 
(B) for g n s=y 2) the estimations of Hks are larger than 
H L . That is the result of a known problem in the es- 
timation of entropic quantities which prevents the es- 
timation to be small. The problem arises because the 
symbolic sequences considered are not infinitely long for 
one to realise that there exists a few or only one sym- 
bolic sequence encoding the trajectory. For example, a 
long periodic orbit would be encoded by a series of short 
symbolic sequences making the estimation of Hks to be 
positive instead of zero as it should be. 



VIII. SYNCHRONISATION (AND 
DESYNCHRONISATION) VERSUS INHIBITION 
(AND EXCITATION) VERSUS INFORMATION 

To understand the relation between synchronisation 
(desynchronisation) and inhibition (excitability), when 
complete synchronisation is absent we do the following. 
But notice that the following results are based on a con- 
jecture that is currently not demonstrated. 

We calculate the Lyapunov exponents along the syn- 
chronisation manifold, which are just the Lyapunov ex- 
ponents of the network by assuming that all neurons are 



completely synchronous. We call these exponents condi- 
tional Lyapunov exponents and the sum of all the positive 
ones is denoted by He- There are two ways for calculat- 
ing them, either using Eq. (5) or (7), Eq. (7) being 
simpler because of the dimensionality of the orthogonal 
vectors employed to calculate the Lyapunov exponents. 
While the use of Eq. (5) requires 3N vectors, each one 
with dimensionality 3N, the use of Eq. (7) requires N vec- 
tors each one with dimensionality 3. Additionally, once 
the function that relates the conditional exponents of two 
mutually coupled neurons with g n and gi is known, then 
one can calculate this function for all the conditional ex- 
ponents of larger networks as long as Eqs. (4) and (7) 
can be rescaled. 

We can then classify these neural networks into 2 types. 
The types UPPER or LOWER. More specifically, 

H c {N,g n ,gi) > H L (N,g n , gi ), UPPER (13) 
H c {N,g n , 9l ) < H L (N,g n , gi ), LOWER (14) 

To understand what He and Hl exactly mean and the 
reason for such a classification, notice that the networks 
here considered admit a synchronous solution. This syn- 
chronous solution might be unstable (an unstable sad- 
dle) and typical initial conditions depart from the neigh- 
bourhood of the synchronous solution and asymptotically 
tend towards a stable solution, the chaotic attractor. 
This attractor describes a network whose nodes are not 
synchronous. In such a situation, the network admits 
at least two relevant solutions: a stable desynchronous 
one (the chaotic attractor) and an unstable synchronous 
one (the synchronisation manifold). While He can be 
associated with the amount of information produced by 
the unstable synchronous solution, Hl can be associ- 
ated with the amount of information produced by the 
desynchronous chaotic attractor. If the complete syn- 
chronous state is stable, then, He = Hl, and the net- 
work in Eq. (2) possesses only one stable synchronous 
solution, for typical initial conditions. The nomencla- 
ture in Eqs. (13) and (14) comes from the fact that 
if H c (N,g n ,gi) > H L (N,g n ,g t ) then, H c is an upper 
bound for Hl, otherwise it is a lower bound [29]. 

Assume now that the more information a network pro- 
duces, the more desynchronisation is observed among 
pair of neurons [29, 30]. If H c (N,g n ,gi) > H L (N,g n ,gi) 
(UPPER), then H L (N,g n ,gi) is limited. As a con- 
sequence, the production of information in the net- 
work is limited and therefore the level of desynchroni- 
sation is small. On the other hand, if Hq{N, g n , gi) < 
H L (N, g n ,gi) (LOWER), then H L (N, g n , gi ) can be large 
implying a large level of desynchronisation. Another way 
of understanding the relationship between synchronisa- 
tion and information is by using a result from Ref. [29], 
which shows that for two coupled maps (but this result 
is trivially extended to networks), the largest transversal 
conditional exponent, when the maps have a LOWER 
character, is larger than this exponent for when they 
have an UPPER character. Since this exponent provides 
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a necessary condition for the stability of the synchroni- 
sation manifold, it can be interpreted as a measure of 
the level of desynchronisation in the network. The larger 
this exponent is, the more desynchronous the network is. 
Therefore, UPPER networks should have neurons more 
synchronous than LOWER networks. 

If H c (N,g n , 9l ) > H L (N,g n , gi ) (UPPER), the 
synapse forces the trajectory to approach the synchro- 
nisation manifold and, as a consequence, there is a high 
level of synchronisation in the network. On the other 
hand, if H c (N,g n , 9l ) < H L (N,g n , 9l ) (LOWER), the 
synapse forces the trajectory to depart from the synchro- 
nisation manifold and, as a consequence, there is a high 
level of desynchronisation in the network. 



(A) (B) (C) (D) 
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FIG. 7: [Color online] Gray regions (green online) indicate 
(g n ,gi) values for which He > Hl (UPPER) and black re- 
gions indicate (g n , gi ) values for which the complete synchro- 
nisation state is stable, in excitatory networks (A-D) and in- 
hibitory networks (E-H). The networks considered in (A-D) as 
well as in (E-H) have the parameters shown in Fig. l(A-D). In 
(B-D) and (F-H) the horizontal axis represent gi(N)\-y 2 (N)\/2 
and the vertical axis kg n . Gray points (green online) appear- 
ing on black regions represent synaptic strengths for which 
in fact one has He = Hl, but numerically we obtain that 
He = Hl + e, with e being a very small positive constant. 

One can check that in Fig. 7, which shows as gray, the 
parameter regions for which He > H L and as black the 
parameter regions for which the synchronisation mani- 
fold is stable and there is complete synchronisation (and 
therefore, He = H L ) for typical initial conditions. Gray 
points appearing on black regions represent synaptic 
strengths for which in fact one has He = H L , but numer- 
ically we obtain that He = Hl + e, with e being a very 
small positive constant. Typically, neurons coupled via 
an excitatory synapse [(A-D)] present a LOWER char- 
acter while via an inhibitory synapse [(E-H)] present an 
UPPER character. 

This classification is also important because as it was 
shown in Ref. [29] , once two coupled neurons are UPPER 



(or LOWER) there is always a synaptic strength range 
for which a large network is UPPER (or LOWER). And 
these synaptic strength ranges can be calculated using 
the rescalings in Eqs. (11) and (12). 

In Figs. 7(B-C) and 7(F-H), we show that the UPPER 
and LOWER character of two mutually coupled neurons 
is preserved in networks composed by a number of neu- 
rons larger than 2, if one considers the rescalings of Eqs. 
(11) and (12). This result is of fundamental importance, 
specially for synaptic strengths that promote the net- 
work to have an UPPER character because it allows us 
to calculate an upper bound for the KS entropy of larger 
networks by knowing the value of He for two mutually 
coupled neurons. Such a situation arises for inhibitory 
networks for a large range of both synaptic strengths. 
One finds an UPPER character in excitatory networks 
for a small value of the chemical synapse strength. 

The electrical synapse favours the neurons to synchro- 
nise. As a consequence, it is expected that networks with 
neurons connected exclusively by electrical synapses are 
of the UPPER type. This can be checked in all figures 
for when g n ~0. 

We are currently trying to prove the conjecture in Ref. 
[29] by studying the relationship between the stability 
of unstable periodic orbits [34] embedded in the attrac- 
tors appearing in complex networks and the stability of 
the equilibrium points. All the equilibrium points of a 
polynomial network can be calculated by the methods in 
Refs. [31-33]. 

IX. UPPER BOUND FOR THE RATE OF 
INFORMATION 

According to Ruelle [28], the sum of all the posi- 
tive Lyapunov exponents is an upper bound for the 
Kolmogorov-Sinai entropy [20]. Therefore, whenever 
H C (N) > H L {N) (UPPER) it is valid to write that 

H C {N) > H KS (N) (15) 

where Hks{N) denotes the Kolmogorov-Sinai entropy of 
a network composed of N neurons. 

As we have previously seen, the UPPER character of 
two mutually coupled neurons is preserved in the special 
larger networks here studied. In addition to this, if the 
positive conditional exponents of two mutually coupled 
neurons are known for a given g n and gi, allowing us to 
calculate H C [N = 2, g n (N = 2), gi(N = 2)], then one can 
calculate the positive conditional exponents of a network 
with N neurons, Hc[N, g n (N), gi(N)]. In other words, if 
the ratio of information production of two mutually cou- 
pled neurons that have equal trajectories, He{N = 2), 
is known and the neurons have an UPPER character, 
one can calculate the upper bound for the ratio of in- 
formation production in larger networks, as long as Eqs. 
(4) and (7) can be rescaled. Therefore, in UPPER net- 
works connected simultaneously with electrical and in- 
hibitory chemical synapses we can always calculate an 
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upper bound for the rate of information production in 
terms of this quantity in two mutually coupled inhibitory 
neurons. 

Consider two mutually coupled neurons. Denote 
Xi(N — 2,g n ) as the sum for the positive Lyapunov 
conditional exponents associated with the synchronisa- 
tion manifold for a chemical synapse strength g n and 
A 2 (iV — 2,g n ,gi) as the sum of the positive Lyapunov 
exponents associated with the only one transversal di- 
rection for a chemical synapse strength g n and an elec- 
trical synapse strength gi. Remind that Ai and A 2 arc 
calculated using Eq. (7) for the index j = 1 and j = 2, 
respectively. 

Now, consider a network formed by N neurons. Using 
similar arguments than the ones presented in Sec. V and 
based on the conjecture proposed in [29], the value of the 
synapse strengths gi(N), g n (N) for which the exponent 
Ai (N) has the same value of Ai (N — 2) can be calculated 
by 



For very large networks that are very well connected, 
gi(N) and g n (N) will be very small, since k and N are 



9n(N) 



9n(N = 2) 

k 



(16) 



and the value of the synapse strengths gi(N), g n (N) 
for which the sum of the positive conditional exponent 
X w (N 7 g n ,gi) (for w > 2) has the same value of A 2 (7V = 
2,g n ,gi) can be calculated by 



9n(N) = 

9l(N) = 



9n(N = 2) 
k 

9l (N = 2)\ l2 (N = 2) 



(17) 
(18) 



Denote \™ ax (N = 2) and X™ ax (N = 2) as the maxi- 
mal values of X\(N = 2,g n ) and X 2 (N = 2,g n ,gi) with 
respect to g n and gi. 

As an example of how to use Eqs. (16), (17) and (18) 
in order to calculate the upper bound for the rate of 
information produced in the network, we consider that 
the neurons in the network with TV nodes are coupled via 
electrical and excitatory chemical synapses in an all-to-all 
configuration (topology II), then k = N— 1, \j w (N)\ = N 
and | 72 (AT = 2)| =2. ' 

Now, we search for a synapse strength range for which 
two mutually coupled neurons have an UPPER character. 
For example, let us say the range gi(N = 2) 6 [0, 1] and 
g n (N = 2) e [2,10], in Fig. 7(E), for two inhibitory 
mutually coupled neurons. 

From Eqs. (17) and (18), as long as the network with 
N nodes has g n (N) < ^^rry and ^ <gi(N) <±, then 
\™ ax (N) = X^iN = 2) and \™ ax (N) = Xf ax (N = 2), 
and therefore for this synapse range, the maximum of 
H c is 



max [# c (iV, 9n , gi )\ = X? ax (N = 2) + (N-l)X™ ax (N = 2) 

9n,9l 

(19) 

Notice that Eq. (19) is valid to any network topology as 
long as Eqs. (4) and (7) can be rescaled. 



large. As a consequence, A™ 
equal, and we can write 



max[ff c (JV, g n ,9i)} 



A? 



- , since neurons are 



NX% ax {N = 2) (20) 



which means that the rate of information produced by 
large UPPER neural networks whose neurons are highly 
connected has an upper bound that increases linearly 
with the number of neurons. A similar result is ob- 
tained when the neurons are connected with only elec- 
trical synapses [29]. 



X. CONCLUSION 

We have studied the combined action of chemical and 
electrical synapses in small networks of Hindmarsh-Rose 
(HR) neurons in the process of synchronisation and on 
the rate of information production. 

There are mainly two scenarios for the appearance of 
complete synchronisation for the studied inhibitory net- 
works. If the chemical synapse strength is small, the 
larger the chemical synapse strength used the larger the 
electrical synapse strength needs to be to achieve com- 
plete synchronisation. Otherwise, if the chemical synapse 
strength is large, complete synchronisation appears if the 
electrical synapse strength is larger than a certain value. 
In the studied excitatory networks both synapses work in 
a constructive way to promote complete synchronisation: 
the larger the chemical synapse strength is the smaller the 
electrical synapse strength needs to be to achieve com- 
plete synchronisation. 

When neurons connect simultaneously by electrical 
and chemical ways, there is an optimal range of synap- 
tic strengths for which the production of information is 
large. For strengths larger than values within this opti- 
mal range, the larger the electrical and chemical synaptic 
strengths are the smaller the production of information 
of coupled neurons. 

In the absence of complete synchronisation, it is in- 
tuitive to expect that excitatory networks have neurons 
that are more desynchronous while inhibitory networks 
have neurons that are more synchronous. This intuitive 
idea can be better formalised by understanding the rela- 
tionship between excitation (inhibition) , synchronisation 
(desynchronisation) and the rate of information produc- 
tion. For that we classify the network as having an UP- 
PER or a LOWER character. In a UPPER (LOWER) 
network, the sum of all the positive Lyapunov exponents, 
denoted by Hl, is bounded from above (below) by the 
sum of all the positive conditional Lyapunov exponents, 
denoted by He, the Lyapunov exponents of the synchro- 
nisation manifold and the transversal directions. Net- 
works that have neurons connected simultaneously by 
inhibitory chemical synapses and electrical synapses can 
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be expected to have an UPPER character. In such net- 
works, one should expect to find synchronous behaviour, 
since the synapses force the trajectory to approach the 
synchronisation manifold. On the other hand, networks 
whose chemical synapse are of the excitatory type might 
likely have a LOWER character. In such networks one 
should expect to find desynchronous behaviour since the 
synapses force the trajectory to depart from the synchro- 
nisation manifold. 

Notice that H L (N) can only be numerically obtained 
whereas Hc{N) can be calculated from the conditional 
exponents numerically obtained for two mutually cou- 
pled neurons that have equal trajectories. For UP- 
PER networks, H C {N) > H L (N), and by Ruelle [28] 
Hl{N) > Hks{N), where Hks is the Kolmogorov- Sinai 
entropy, the amount of information (Shannon's entropy) 
produced by time unit; we have then that Hq is an upper 
bound for Hks(N). That can be advantageously used in 
order to calculate the rate of information produced by 
a large network, composed of N neurons by using only 
the rate at which information is produced in two mu- 
tually coupled neurons that are completely synchronous 
and have equal trajectories. 

We have worked with idealistic networks. However, our 
results can be extended to more realistic networks [12]. 
For UPPER networks, our numerical results show that 
more realistic networks constructed with non-equal nodes 
(or networks of equal nodes but with random synapse 
strengths [30]) have Hl smaller than the networks with 
equal nodes. Therefore, even though networks with equal 
nodes might not be realistic, their entropy production per 
time unit is an upper bound for the entropy production 
of more realistic networks. 
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XI. APPENDIX 
A. A lower bound for the KS entropy 

Imagine a 2D chaotic system as the one studied in Ref. 
[11] [Eqs. (5) and (6)]. Following the same ideas from 
there, the KS entropy of two coupled maps with variables 



x a and x" can be estimated from the Shannon's entropy 
of the probabilities that a trajectory point makes a given 
itinerary in the phase space (x a 1 x"), divided by the time 
interval for the trajectory to make that itinerary. 

In practice, calculating the Shannon's entropy [18] for 
all possible itineraries on the phase space (x a ,x^) of a 
chaotic trajectory is equivalent to calculating the joint 
entropy between the probabilities of finding a point fol- 
lowing simultaneously an itinerary along the variable the 
variable x a and another itinerary along the variable x@ . 

Since we are unable to make a high resolution parti- 
tion of the phase space (nor we do not know the Markov 
partition) in the neural networks studied in this work, we 
estimate a lower bound for the KS entropy by calculating 
the joint entropy between symbolic sequences encoding 
the trajectory. Such calculation of probabilities involve 
large matrix operations and for that reason we restrain 
ourselves to the calculation of the joint entropy between 
two neurons. 

It is a lower bound due to two reasons. The first one 
is because the entropy will be measured considering the 
probabilities of occupation of a projected trajectory in a 
subspacc of the network. The second one is because we 
calculate the entropy considering the probabilities of bi- 
nary symbolic sequences and obviously a binary sequence 
may contain much less information than the content of a 
continuous signal [19]. 

In the following, we show in more details how this es- 
timation is done. The way we encode the trajectory is 
partially based on the time encoding proposed in Ref. 
[30]. 

Given two symbolic sequences S\ and S%, generated by 
neuron 1 and 2, respectively, a lower bound for the KS 
entropy can be estimated by 

Hi ow = — -H(Si; S2) (21) 

(v 

with iJ(S , i;5 2 ) representing the joint entropy between 
the symbolic sequences S± and S2. To create the sym- 
bolic sequences, we represent the time at which the n-th 
maxima happens in neuron 1 by T™, and the time in- 
terval between the n-th and the (n+l)-th maxima, by 
ST™. A maxima represents the moment when the action 
potential reaches its maximal value. The quantity (r) 
represents the average time between two spikes. We then 
encode the spiking events using the following rule. The 
z-th symbol of the encoding is a "1" if a spike is found in 
the time interval [zA, (i + 1)A[, and "0" otherwise. We 
choose A £ [min (ST™), max (ST™)] in order to maximise 
Hi ow . Each neuron produces a symbolic sequence that is 
split into small non-overlapping sequences of length L=8. 
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